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Introduction. 
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.  This  report  is  a  summary  of  the  state  of  the  art'. and  of 
trends  in  development  in  the  field  of  quality  control  in 
complex,  highly  automated  production  systems.  Special  em¬ 
phasis  is  given  to  the  development  and  use  of  modern  optical 
sensor  systems  and  components  in  industrial  metrology. 

The  report  compares  and  contrasts  accomplishments  in  the 
field  in  both  the  USA  and  the  Federal  Republic  of  Germany. 
The  comparison  was  made  possible  by  an  extensive  visit  to 
the  USA  arranged  by  the  OS  Air  Force  European  Office  of 
Aerospace  Research  and  Development  (EOARD)  at  the  request 
of  the  Air  Force  Integrated  Computer  Aided  Manufacturing 
Project  (ICAM) .  The  authors  visited  Air  Force,  industrial 
and  government  laboratories  engaged  in  research  in  modern 
quality  control  techniques.*  This  visit  is  summarized  in  the 
next  section  of  this  report..  '•  * 


Sections  3  and  4  discuss  trends  in  development  in  the  field 
of  quality  control  and  quality  assurance  and  activities 
respectively , at  the  Institut  fur  Produktionstechnik  und  Auto- 
matisierung  ( IPA) . Section  5  contains  detailed  examples  of 
visual  inspection  systems  for  quality  assurance. 
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The  IPA  in  Stuttgart,  West  Germany,  is  an  Institute  of  the 
Fraunhofer  Society.  Applied  research  in  the  field  of  pro¬ 
duction  technology  and  organization  is  done  with  a  staff  of' 
about  two  hundred  people.  Support  for  this  research  comes 
either  from  direct  contract  v/ith  industry  or  is  sponsored  by 
public  funding. 


Introduct i on 


One  of  the  main  working  areas  is:  the  application  o.f  industrial 
robots  in  highly  flexible  production  systems.  A  lot. of  per¬ 
ipheral  problems  are  significant ' for  the  integration  of  such 
systems;  human  aspects  as  well  as  economic  considerations 
have  to  be  dealt  with. 

The  flexibility  of  an  industrial  robot  is  limited,  by  the 
ability  of  its  sensors  to  react  to  changing  working  conditions 
The  wide  range  of  optical  sensors  together  with  sophisticated 
electronics  show  the  way  for  solving  such  problems,  toda^ . 
Furthermore ,  the  basic  components  of  soft-  and  hardware ■ of 
these  modules  are  used  in  the  field  of  measurement  and  qua.1 
ity  control.  The  automation  of  visual  inspections  seems  es¬ 
pecially  now  to  be  possible  under  economic  conditions. 
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Journey  Report 


On  behalf  of  EOARD,  members  of  the.  I  PA  s-fesff  went  on  a  jour¬ 
ney  through  the  USA  to  gain's  deeper  insight  rn  the  American 
activities  in  research  and  development  in  the  field  of  robot 
and  sensor  technology.  During  this  journey,  they  visited  , 
several  industrial  businesses-  and  research  institute^,  not 
all  of  which  were  working  Under  the  ICAM  project.  The  itin¬ 
erary  is  shown  below: 

ITINERARY 


DATE 


6 .  iO 


7.10 

8.10 


9.10 


PLACE  OF  VISIT- 

Materials  Laboratory  (HLTCj 

Wright-Patterson  ADD 
Dayton,  Ohio 

Cincinnati  Milacroh 
Cincinnati,  Ohio 

McDonnell  Douglas  Aircraft  and 
Automation 
St.  Louis,  Missouri 

Chrysler  Corporation 
Huntsville  Electronics  Division 
Huntsville,  Alabama 
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General  Dynamics 
Fort  Worth,  Texas 


Stanford  Research.  Institute 
Menlo  Park,  California 

and 

Unimation  West 

Mountain  View  ,  California 


Stanford  University 

Dept,  of  Mechanical  Engineering  and 
Dept,  of  Computer  Science,  Artifici 
Intellicence  Laboratory 
Palo  Alto,  California 

Jet  Propulsion  Laboratory 
Pasadena,  California 


National  Bureau  of  Standards 
Washington,  D.C. 


Lt.  G.-E. 
nautical 
duction 


Mayer  of  the 
Laboratories , 


Materials  Laboratory,  AF  Wright  Aero 
Wright-Patterson  AFB  rave  an  intro- 


to  the  ICAM  project. 


connection,  he  also 


In  this 
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emphasized  the  cooperation  with  General  Dynamics  (Fort  Wor^h, 
Texas)  and  McDonnell  Douglas.  Automation  Corporation  (ot*  Louis, 
Missouri)  ,  v/here  aircraft,  parts  are  drilled  by  Cincinnati 
Mi  lac  r  oh  robots  or  riveted  in  manufacturing  cells.-  Specially 
developed  sensors  /!/  and  the  use  of  several  TV  cameras  for 
part  identification  are  some  characteristics  of  the  advanced 
state  of  development  of  American  robotic  systems. /2/ . 

The  handling  of  parts  using  very . simple 'sensor  equipment  was 
demonstrated  at  Cincinnati  Milacron..  Merton  D.  Corwin,  manager 
of  the  Department  of  Robot  Research  and  Development  informed 
in  detail  about  future  trends  in  this  area.  Cincinnati's  new 
machine,  the  T^R^,  has  a  very  comfortable  continuous  path  con¬ 
trol-  with  a  lot  of  software  diagnostic  tools  ror  supervision. 
Transformation  of  work-piece  coordinates  to  robotic  coordinates 
is  fully  achieved  by  the  control  processor.  Significant  fix- 
ooints,  e.g.  the  tool  center,  can  be  shifted  anywhere  within 
the  workspace . 

Chrysler  Corporation,  Huntsville  Electronics  Division,  Alabama, 
demonstrated  some  optical  sensors  and  sensor  systems,  one  of 
which  was  fully  integrated  in  a  production  line  This  system 
was  being  used  for’  inspecting  PCB's.  A  two-dimensional  laser 
scanner  detects  the  presence  of  absence  o  a  leac.  in  the  respec 
tive  boring.  The  geometry  of  the  bore  pattern  is  programmed 
via  a  "  teach-in*1  procedure.  Errors,  if  any,  are  given  tne  ao^* 
prourxate  number  from  the  bore  pattern.  These  numbers  are  also 
made  available  for  further  use. 

Another  system  was  at  the  moment  in  the  testing  phase.  Its 
purpose  is  to  scan  large  parts.  The  advantage  of  this  system 
is  that  it  can  be  used  with  relatively  simple  data  processing 
equipment.  It  should  be  noted  that  both  these  projects  are 
not  supported  by  ICAM. 

At  General  Dynamics,  Fort  Worth,  Texas,  two  ICAM- funded  projects 
are  in  application,  another  one  Is  under  consideration  and  will 
be  carried  by  General  Dynamics  itself. 

A1 1  three  arc  manufacturing  cells  for  drilling  of  aircraft  parts 
with  different  sizes  and  shapes  using  Cincinnati  Milacron  robots. 
Due  to  the  lack  of  position-  accuracy  required  for  high  precision 
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drilling,  mechanical  aids  are  used  for  guiding  and  fixing  the 
drilling  tool  as  well  as  other  gripper- tool'  combinations  .which ... 
can  be  interchanged  autoraati  ca-lly . 

Concerning  investments,  General  Dynamics  dees  noc  seem-  to  be 
a  typical  applier;  the  -pay-off  rates  accepted  might  not  be 
transferred  to  other  branches  of  industrial  proauction  with- 
out  further  considerations. 

At  Stanford  Research  Institute,  Menlo  Part,  California,  David 
Nitzan,  Program  Manager  for  Industrial  Automation  gave  a  brief 
overview  of  future  tasks  and  projects.  The  automation  ct 
visual  inspection  is  one  the  most  significant  topics  for.  the 
coming  years. 

At  Stanford  University,  robots  are.  used  for  the  complete  assem¬ 
bly  of  flashlights,  sprinkler  heads,  etc.;  these  robots  make 
direct  responses  to  speech  commands  /3/,  and  they  permit  mani- 
pulations  while  moving. 

Changes  in  reference  point  and  in  position,  which  may  be  caused 
by  collision  of  the  robot  or  maintenance,  are  corrected  auto¬ 
matically  with  "Puma"  of  Unimation  West  (Mountain-  View)  .  The 
robot  can  also  be  adapted  in  its  movements  to  a  running  assem¬ 
bly  line,  and  it  can  thus  move  in  synchronism  with  the  assembly 

line  , 

The  progress  made  by  NASA's  Jet  Rropulsion  Lab  (Pasadena,  Cali¬ 
fornia)  ,  where  research  and  development  activities  are  made 
in  connection  with  space  operations,  deserves  special  mention. 
One  of  the  interesting  developments  is  a  manipulator  mounted., 
on  a  vehicle,  equipped  with  force  and  torque  sensors  and  opti¬ 
cally  linked  with  the  outside  world  via  two  camera  systems, 
ar.d  the  "voice  control"  for  acting  on  a  "space  shuttle"  con¬ 
troller  by  acoustic  means.  A  microprocessor-controlled  device 
for  detection  of  the  contacted  surfaces  to  be  used  within  the 


industrial  robots  was  demonstrated;  this  device 


gripper  zone  oi 

also  shows  the  gripping  forces  active  at  a  giver,  moment  m  a 
contact  matrix. 

The  problem  of  the  gripper-interface  standardization,  which  is 


.Li 
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not  only  specific  to  the  USA,  is  dealt  .with  hv  the  National 
Bureau  of . Standards  (NB5,  Washington,  D.C.);  this  Bureau  has 
been  extremely  successful  in  the  development  of  gripper  sys¬ 
tems  (as  e.g.  in  Puma  robots) . 


References  relating  to  the  text: 


/!/  Nltzan ,  D.  :  Proceedings  of  MX DC ON/ 7 9 ,  Chicago,  IL, 

6-. -8.  November  197  9 

/2/  Eastwood,  M.A.:  McDonell  Douglas  Automation  Co. ,  St.  Louis, 

September  8,  1980;  Robotic  System  for  Aero¬ 
space  Batch  Manufacturing 

IEEE  Transactions  on  Computers,  Vol.  <2-25, 
No.  12,  December  1976 


/ 3 /  Nitzan,  D. ; 
Rosen ,  C. A. : 


/4 /  a)  Freese ,  F. ; 

Geise,  P. :  Chrysler  Corporation/  Huntsville 

Electronic  Division?  A  Laser-scanner 
for  automatic  inspection  of  printed 
wiring  boards 

b)  Geise,  P. ; 

George,  E. :  Chrysler  Co.,  Huntsville  Electronic 

Division?  Axle  Shaft  Optical  Inspec¬ 


tion  System 
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Within  the  US,  the  ICAM  Program  is.  the  most  comprehensive 
effort  to  improve  productivity  through  utilization  of  ad 
vanced  technology..  The  following  is  a  description  of  the 
ICAM  Program  as  perceived  by  the  autaors. 

Influenced  by  the  rapidly  changing  conditions  of  science 
and  technology  and  the  steadily  growing  competition  from  ab¬ 
road,  the  ICAM  project  was  created  under  the  leadership  of  the 
USAF  and  with  the  support  of  the  United  States  Department  of 
Defense.  When  the  project  began  in  1977,  the  starting  point 
was  a  cooperation  among  private  and  public  institutions  and 
American  industrial  businesses  aiming  at  the  rapid  advancement 
of • scientific  and  technological  progress  and  its  introduction 
into  application-related  practical  work. 

It  is  quite  natural  in  such  a  complicated  venture  that  the 
first  task  was %  to  clarify  questions  of  organization  and  to 
lay  down  a  uniform  course  of  action.  This  included  the  stan¬ 
dardization  of  programming  languages. and  development  of  an 
architecture  of  manufacturing  which  are  used  by  everyone  in- 

volved  in  ICAM.  •  & 

in  the  foreground  are  such  factors  as  economy  in  the  design 

and  development  of  complex  flexible  production  systems;  the 
replacement  of  man,  whose  capability  is  restricted  with  re¬ 
spect  to  specialized  sensor  problems,  by  highly  specialized 
sensors;  the  reduction  in  the  implementation  period  of  co.iq 
tible ,  standardized  methods;  etc.,  where  these  factors  are 
regarded  as  ideal  goals.  Particularly  in  the  production  of 
military  products  and  the  problems  that  exist  in  this  field 
(small  batches,  frequently  changing  products,  greau  accuracy 
and  reliability,  etc.),  improvements  are  to  be  obtained  on 

these  lines. 

The  field  of  sheet-metal  working  has  been  chosen  to  serve 
as  an  example  in  demonstrating  the  effectiveness  of  ICAM 
technology.  The  modular  components*  developed  along  this  line 
will  then  be  ready  for  use  in  the  ICAM  factory,  as  It  has 
been  called. 
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Op  to  the  end  of  1980,  progress  mainly  emerged  in  the  so'  • 

ware  field!  This  involved  the  sof  tware  analysis  of-  exis  •  ir.g  . 
programs  and  their  potential  integration  into  the  iCSl'jrejee  . 
The  programs  are  modular  in  structure,  so  than  tic  is, 
tation  of  separate  modules  is  made  possible  m  in  m 
industries,  the  main  benefit  is  however  to  be  draw. from  the 
felly  integrated  system.  The  computer  is  used  on  all  1" 
to  support  and  Chech  decisions,  and  it,  for  example,  organises 

jointing  and  machining  cells. 


As  shewn  by  the  ICAM  reports  available,  many  sectors  of  the 
torn  project  have  already  been  included  in  the  current  ih 
vestigations .  Among  the' sectors  still  open  the. .  » .  * - • 
all  one  that  can  also  be  seen  in  conjunction  with  «»« 
ties  going  on  at  IPA,  this  is  the  sector:  Test,  Inspect,  Eva 

luate,  Quality  Assurance. 

U.S/A1R  FORCE  _ 

....  1NFEGRATED  COMPUTER  AIDED  MANUFACTURE 


•  i._  *-hfa  comolcx  problems  of  quality  cotivTO. 

investigations  into  uhe  ccmoie.  L  •  P_derai 

,.,v  assurance  are  given  much  attention  m  the 
and  qualify  assurance  y  .  result  of 

i  -.rifi  it  IPA  in  particular .  As  a 
Republic  m  general  aXid  at  iia  1  .  .  -e  „  lonacr 

h  ^  becoming  scarcer,  it  is  no  longer 

pnnrnv  and  raw  materials  ULLUUU1  J 

energy  ana  ;  hv  CUantity.  A  product  under 

acceptable  to  achieve  quality  bi  i 
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development  must  be.  subjected  to  inspection  prior  to,  during, 
and  after  working  on  it,  so  as  to  keep  rejects  as  low  as  pos¬ 
sible.  The  quality  characteristic  "reliability”  not  only  - 
plays  an  important  .role  in  the  military  field*  although  it  is 
of  greater  significance  there.  It  is  particularly  important 
in  complicated  technical  equipment  where  reliability  is  en¬ 
dangered  by  a  large  number  of  individual  components,,  since 
the  failure  of  a  single  component  may  result  in  the  breakdown 
of  the  total  system. 

For  assuring  the  quality  of  a  product,  it  is  necessary  to 
examine  the  disturbance  variables  that  may  occur  randomly  or. 
systematically- the  human  worker,  the  methods  used  in  pro¬ 
duction,  the  machines  and  the  materials  involved-  as  to.  their 
effects  and  the  frequency  with  which  they  occur .  * 

At  IPA,  research' and  development  work  has  been  performed  in 
the  field  of  noncontact  sensors,  in  particular  optical  sen¬ 
sors,  for  a  couple  of  years-;  among  other  things,  this  may  in 
volve  the  detection  of  surface  imperfections  (e.g.  shrink  holes, 
rusty  spots,  defects  and  irregularities  of  materials) ,  the 
measurement  of  geometrical  bodies  (such  as  screws,  bolts, 
pressed  parts)  and  the  determination  of  the  surface  roughness  of 
technical  surfaces. 

To  demonstrate  the  development  in  this  field  in  more  detail., 
some  activities  will  be  discussed  at  length. 
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Optical  measurement  of  surface  rougnne.^s. 

•  .  .  .  .  rt  t-^e  f  ield  of  optical  roughness  • 

The  research  activities  ih  t».e  tie m  r  ' 

measurement  on  workpiece  surfaces  have  experienced  a  scag-- 

goring  rate  of  development  since  1972.  Sensors,  and  par„cu 

larly  those  developed  in  Japan  have  been  able  to  produce 

or „  ehnn  floor  has  not  however 
good  results.  Their  use  on  the- shop  -io 

materialized  up  to  the  present  time. 

The  investigations  carried  out  in  this  field  at  ir*  from 

1975  to  1979  produced  comparable  results  (Figure  1).  In 

1979/1980  a  somewhat  modified  new  technique  for  optical 

roughness  measurement  was  eventually  dei/eiop-d , 

likely  that  this  new  technique  will  soon'  be  used  on  the 

shop  floor,  in  this  setup,  called  -'white-light-method 

there  is  a  white-light  bulb  (halogen  bulb),  whose  light  .falls 

on  the  surface  to  be  measured.  The  back-scatte.e  ^ 

deflected  via  a  beam  splitter  and  projected  on  the 

level  by  the  image  forming  section.  An  adaitiona.  con 

UgKt  overbuy  Uneurixes  th.  cbnracterfstic  curve  and  barge- 

iy  reduces  the  declining  trend  in  the  "peak-tc  valley 

pattern1'  (Figure  2)  * 

On  the  out-of-focus  level,  the  optical  information  of  the 
surface  to  be  measured  is  converted  into  electronic^  rn  o. 
nation  through  a  photosensitive  sensor  (as  e.g.  a  pno  ° 
multiplier,  or  a  semiconductor  operational  amplifier  ^ 
light-sensitive  diode) .  In  a  proper  statistical  evalu 
of  the  current  or  voltage  values  carried  out  by  a  cu.. 

(KP  9815A,  COMMODORE  PET)  linked  into  the  system  x.  xs 
possible  to  make  clear  assignments  of  the  measure 
values  to  the  surface  parameter  (RQ,  Rq) r 

,p,rcbi  work  in  this  field  is  continued 
Ivhen  the  current  rescarc.  • 

t  c  every  likelihood  t.ist  a 
in  an  intensive  manner,  there  -  Y  . 

prototype  of  this  roughness,  sensor  will'  be  implements  .  in 
1981  /198.2  . 
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Figure  1  Speckle  contrast  method  for-  measuring  the 
roughness  on  workpiece  surfaces 
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constant light  overlay 


Figure  2  :  whi te -'Light- -Me fchod  for  measuring  the  rough¬ 
ness  on  workpiece  surfaces.  Characteristic • ^ 
curves  measured  (TJ  -I-/I  '  constant  iight  overlay) 
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Linear  and  area  arrays  of  photodiodes 

The  advancing  development  in  semiconductor  technology 
made  available  optical  sensors  arranged  in  linear  form  or 
in  arrays.  These  sensors  can  be  used  for  optical  quality 
control. 

Screw-bolt  inspection  . 

This  inspection  uses  the  linear  diodes,  with  the  diodes 
being  transversely  arranged  to  the  screw  bolt's  axis.  If 
the  screw  bolt  is  moved  in  axial  direction,  the  individual 
diodes  arranged  in  a  row  will  be  illuminated  or  dimmed,, 
depending  on  the  contour  involved.  By  counting  the  bright/ 
dark  elements,  a  length  assignment  can  be  made,  depending 
on  the  optical  set-up  involved. 

The  values  obtained  in  the  individual  line  scans  are  then 
linked  and  further  processed  by  a  POP  11/03  computer,  which 
determines  the  characteristic  features  of  the  screw  bolt 
such  as  pitch,  thread  angle,  completeness, etc *  in-  accordance 
with  the  objectives  prescribed.  The  software  permits  the 
entry  of  sample  pieces  by  the  "teach-in"  method,  and  a  com¬ 
parison  can  then  be  made  with  actual  parts. 

At  the  present  time,  the  state  of  development  and  the  relia¬ 
bility  of  the  linear  diodes  still  leave  the  array  diodes 
somewhat  in  the  background.  But  arrays  will  be  used  for 
tasks  in  the  future,  which  so  far  have  exclusively  been  re¬ 
served  for  linear  diodes. 

This  development  work  is  performed  by  IPA  in  parallel,  lie. 
both  linear  and  array  diodes  are  used  to  solve  problems , 
so  that  linear  diodes  can  immediately.be  replaced  by  arrays, 
if  the  technological  development  goes  on  at  this  rapid  rate. 
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:  Linear  diodes  used  in  screw-bolt  inspection 
(principle) 


Figure  3 
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:  Measuring  signals 


in  screw-bolt  inspection 


Figure  4 
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Figure  5  :  Signal  processing  in  screw-bolt  inspection 


Ficure  6  :  Diode  array  used  in  screw-bolt  inspection 

—  Monitor  image  of  a  thread  (100x100  diode  matrix) 
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Piston-ring  inspection 

Another  field  of  application  of  the  linear  diode  earner's 
is  the  optical  inspection  of  surface  characteristics  on- 
piston  rings.  The  purpose  of  this  inspection  is  the  detec¬ 
tion  of  surface  defects  {shrink  holes t  rusty  spots,  materia 
eruptions  etc.)  which  exceed  a  predetermined  value.  The  se¬ 
quence  of  measurement  is  as  follows: 

The  clamped  piston  ring  is  turned  via  a  movable  table  (ro¬ 
tary  table) .  The  scan  start  is  determined  on  the  basis  of 
the  signals  coming  from  the  linear  diodes,  and  measurements 
will  be  made  throughout  the  predetermined  angle  range. 

The  individual  error  signals  thus  obtained  will  be  accumu¬ 
lated  and  an  error  message  will  occur  as  soon  as  the  tole¬ 
rable  cumulative  error  is  exceeded.  Piston  rings  falling 
below  and  exceeding  the  tolerance  limits  will  be  separated 
by  connected-in  electronically  controlled  mechanical  equips 
ment,  and  the  piston  rings  can  thus  be  detected  as  "go"  or 
"no  go". 


Figure  7  :  Experimental  set-up  for  piston-ring  inspection 


Figure  8 


Determination  of  measurement  positions  for  a 
measurement  cycle 


surface  defect 


Figure  9  :  Principle  of  covering-  the  "defect  areas" 


Figure  10  :  Evaluating  the  video  signal  for  determining 

the  "width  of  defect" 
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Laser  scanner 

The  laser  scanner  is  a  device  which  is  suitable  for  measur¬ 
ing  large  dimensions  (in  the  mater  range)  at  great  accuracy 
(in  the’  range  of  100  un)  .  In  these  measurements,  moving 
pacer  webs  or  metal  lines  can  be  inspected  for  holes,  irre¬ 
gularities,  inclusions,'  etc.  Special  peripheral  computers 
permit  flexibility  in  the  selection  of  tolerance  limits, 
so  that  an  adaptation  to  the  problems  involved  is  possible 
without  any  major  difficulties. 


fan-out  optics  plane  mirror 


:  Laser  scan  system  for  measuring  large  size 
surfaces  (metal  line  and  paper  web  surfaces) 


Figure  1 1 


marking  selection  gate  storage 

device 


Figure  12  :  Signal  evaluation  in  the  laser  scanne 
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Visual  Inspection  ana  metrology 

Typical  visual  inspection  tasks  are.  e.g.  completeness  tests 
during  assembly  or  during  production,  surface  tests,  and  tests, 
to  determine  the  shape  and  position,  to  the  extent  these 
can  still  be  recognized  by  visual  means. 

The  better  the  quality  of  the  workpieces  to  be  tested  and 
the  less  frequent  the  sight  inspector  experiences  a-  success 
in  that  he  finds,  a  defect,  the  more  likely  observation  er.rors 
will  occur. The  efficiency  of  the  visual  inspector  wall  reach 
a  maximum  degree  of  about  9o%,if  he  detects  a  defective  pea. t 
every  2o  seconds  on  the  average. His  efficiency  will  decrease 
to  approximately  6o%,if  too  little  demand  or  excessive  demand 
is  made  on  him. 


But  it  is  not  only  the  monotony  linked  with  visual  inspection- 
work  stations  that  makes  visual  inspection  tasks  pro¬ 
blematic  in  the  quality  assurance  environment, 

but  also  miniaturization,  particularly  in  the  field  of-  elec¬ 
tronics,  and  the  production  speeds  -  these  increasingly  set 
limits  to  visual  inspection. 

Thus , endeavours  to  automate  visual  inspection  will  be  made 
whenever  it  becomes  necessary  to  objectify  the  visual  in 
spection  process,  if  the  characteristics  of  the  visual  in¬ 
spection  tasks  can  no  longer  be  detected  by  visual  means,  or 
if  a  visual  inspection  station  has  to  be  automated  for  rea- 
sons  of  time  or  cost. 

automate  visual  inspection  stations  is 
processing  systems'. 

These  systems  consist  of  an  image  converter  which  takes  the 
form  of  television  cameras  or  diode  arrays,  more  or  less 
convenient  mapped  memories,  a  picture-processing  section, 
and  different  output  facilities.  • 


One  possibility  to 
the  use  of  picture- 
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The  Fraunhof er-Insti  tut  .'fur  Produktionstechnik  und-  Automat x- 
sierung  has  a  number  of  such-  optoelectronic  picture  processing 
systems,,  which  will  be  described  in  the  following:. 

The  knowledge  of  the  advantages  of  telex  i->ion  s/^tcr 
suited  in  the  feet  that  research  and  development  work  in  this 
sector  has  been  heavily  pushed  forward.  The  modern  image  data 
collection  devices,  however,  supply  such  large  volumes  of 
data  that  it  is  no  longer  possible  for  man  to  evaluate  them 
by  manual-visual  means.  Only  the  drastic  price  fall  in  the 
process  computer  market  made  It  possible  to  develop  sensor 
systems  coupled  to  computers,  and  this  at  relatively  reason- 

able  cost. 

The  interactive  picture-processing  system  used  at  IPA  is  to 
serve  as  an  example  in  the  processing  of  the  big  data  volumes 
to  be  handled.  Kith  a  linear  picture  resolution  of.  512  by  51- 
picture  elements,  a  gray-level  resolution  of  256  levels,  and 
a  picture-element  scanning  speed  of  nearly  15  MHz, 
nal  fields  represent  such  an  extremely  big  information  volume 
which  can  only  be  evaluated  by  using  special  image  -processors 
Since  the  operating  speed  of  such  sensor  systems  does  not 
least  of  all  play  a  decisive  role,  and  as  explained  above 
with  a  standardized  volume  of  image  data  to  be  handled,  new 
approaches  had  to  be  found  in  attaining  processing  speeds^ 
that  are  adequate  for  the  applications..  The  development  of 
high-speed  digital  mapped  memories  in  combination  with  pro¬ 
cessors  for  image  preprocessing  now  provides  the  possibility 
to  evaluate  a  picture  scene  within  times  acceptable  in  prac 

tical  work. 

The  tasks  of  such  sensor  systems  are  very  versatile.  It  xs 
therefore  of  great  importance  to  prepare  a  catalog  of  tasks, 
which  permits  the  sensor  system  and  its  configuration  to  be 
allocated  to  the  special  test  problems.  The  systems  for  pre 
ture  processing  necessary  for  the  solution  of.  subh  special 
evaluation  jobs  are  expediently  implemented  through  special 
hardware  and  software  systems ► 
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If  one  investigates  the  trend  of  development  in  the  field-- 
of  television  picture-processing  systems,  it  can  be-  seen 
that  a  modular  design  of  hardware  and  software  is  given 
preference,  because  of  the  wide  range  of  applications.  The 
future  fields  of  application  of  automatically-  operating  tele 
vision  picture-processing  systems  can  be  divided  into  five 
fields  of  application,  with  tasks  allocated  accordingly: 

-  industrial  applications 

-  military  applications 

-  biomedical  applications 

-  remote  reconnaissance  of  planets 

-  special  applications. 

Out  of  this  catalog  of  applications,  industrial  inspection 
problems  will  be  dealt  with  here,  and  in  particular  the 
problems  of  visual  inspection  and  quality  control. 

Two  television  picture-processing  systems  are  available  for 
research  purposes  and  for  carrying  out  jobs  received  from 
industry.  One  picture— processing  system,  the  scanning  j^stJii, 
makes  it  possible  to  take  over  part  of  the  information  from  any 
position  of  the  picture  field  and  to  process  this  information. 
The  interactive  picture-processing  system  is  equipped  with 
a  real-time  mapped  memory,  in  which  a  complete  television 
picture  with  256  gray  levels  per  picture  element  can  be  di¬ 
gitised  and  frozen  in  real  time.  Both  systems  will  be  ex- 
olained  in  more  detail  on  the  following  pages. 
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TV  scanning  system 

This  system  for  picture  processing  consists  of  a  television 
camera  manufactured  by  HAMAMATSU  (Figure  131ano  it  is  ct.arac 
terised  by  its  high  stability,  high  responsiveness,  high 
resolution  11024  x  1024  picture  elements)  and  low  wear  due ■  _ 

to  the  burnt-in  raster.  The  analog  pictorial  data  is  cigiti 
in  the  camera's  control  unit  (Figure  14)  and  fed  into 
process  computer  via  a  parallel  interface.  Furthermore,  t.  e 
camera's  control  unit  also  controls  the  camera,  the  acceptance 
of  data,  it  is  also  possible  to  perform  image-preprocessing 
With  this  unit.  It  is  e.g.  possible  to  improve  the  linearity 
capabilities  of  the  camera  and  to  generate  binary  images  using 
a  gray-level  threshold  that  can  be  manually  selected. 

The  color  convertor  permits  colors  to  be  allocated  to  a  black- 
and-white  picture,  thus  making  it  into  ,  multi  -olor  P 
By  suitable  color  allocations,  the  color  converter  beco . 
valuable  component  in  the  visual  'judgement  of  on  Ob  ■'  c  9 

Particularly  in  the  illumination  of  homogeneous  gray  fiel  s 
and  in  contrast  examinations,  minor  deviations  in  the  3«y 
level  can  only  be  visually  represented  and  perceived  with 

allocation  of  false  colors. 

V 

Real-time  system 

This  system,  manufactured  by  Micro  Consultants  Ltd.  of  type 
"Intellect"  is  capable  of  digitizing  a  video  picture  in 
time,  and  then  storing  it  in  a  semiconductor  memory.  In  this 
form,  the  digital  image  can  be  conditioned  by  the  computer 
and  manipulated  in  any  manner.  The  scanning  frequency  o  •  • 

of  the  8-bit  A/D  converter  does  not  only  ensure  great  sy 
but  also  a  high  reproduction  quality. 


accuracy , 
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14  :  Camera  control  unit 
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The  core  of  the  system  is  the  semiconductor  memory  with  a 
capacity  of'256'times  51?  words  of  8  bits  each  for  storing 
two  fields.-  i.e.  256  lines  with  512  picture  elements  for  each 
line  of  storage  space  are  available  per  fiell,  with  a  re 
lution  of  8  bits  per  element.  Image  line  and  image  dot  arc 
each  defined  by  a  9-bit  address.  Since  the  minicomputer  has 
direct  access  to  each  individual  image  dot,  it  can  take  in¬ 
formation  out  of  the. memory,  read  it  in  again  or  produce  an ^ 
image  with  regenerated  data.  Moreover,  this  makes  the  transfer, 
rate  dependent  solely  on  the  speed  of  the  computer . Read  and 
write  operations  in  the  memory  are  performed  completely  inde- 
pendently  from  one  another,  so  that  both  synchronous-  movements 
(cameras)  and  asynchronous  movements  (e.g.  electron  microscope) 
will  be  possible. 

This  system  permits  numerous  manipulations  to  be  made  undei- 
hardware  and  software  control,  as  e.g.- 

-  relocation  of  the  image  in  memory 

-  special  marking  of  specific  image  sections 

-  image  representation  at  reduced  resolution 

-  image  reversal 

-  writing  two  fields  one  above  the  other 

-  X-Y-coordinate  determination  of  defined  points 

-  color  representation. 


This  system  can 
e.g.  to  a  camera  o 
camera  with  BAS  exit. 


also  be  linked  to  high-precision  cameras f  as 
-F  HAMA  MAT  pU ,  or  to  any  other  television 


Software  development 

At  the  Fraunhofer- Inst itufc  fur  Produktionstechnik  und  Auto- 
matisieruog,  software  for  picture  processing  is  developed  for 
a  wide  range  of  applications. 
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The  most  important  guiding  principle  in  the  development 
of  software  is  a  program  architecture,  that  is  structured 
to  a  great  extent, so* that  the  possibility  in  software, 
development  of  creating  a  modular  software  library  as  a 
universal  application,  can  be  realized. 

At  the  TPA  a  concept  has  been  developed  in  addition,  which 
permits  the  individual  modules  to  be  used  in  the  most 
different  picture-processing  systems.  Thus,  software  for 
geometry  analysis  is  e.g.  used  on  the  somewhat  slower 
operating  scanning  system  as  well  as  on  the  interactive 
picture-processing  system.  Moreover,  it  is  possible  to  em¬ 
ploy  this  software  -when  using  diode  cameras  with  a 
one-dimensional  field  (1024  picture  elements)-  in  connection 
with  a  mechanically  movable  scanning  table.  This  system  is 
also'  used  at  the  IPA.  ■ 

The  processing  of  simple  binary  images  will  not  t>e  aB- 
sufficient  where  tasks  are  mainly  to  be  handled  in  quality 
assurance  and  in  visual  inspection.  In  these  tasks,  grav- 
level  analyses  -  both  statistical  and  discrete  have  to  be 

performed.  I 

4* 

Different  methods  of  finding  threshold  values  are  in 

binarv  picture  processing-  It  is  particularly  in  the  geo¬ 
metry  analysis  where  the  finding  of  a  suitable  gray-level 
threshold  is  decisive. 

Picture-enhancement  programs  are  used  for  correcting  line¬ 
arity  and  individual  shading  effects,  they  can  also  elimi¬ 
nate  faulty  information  or  the  invalidation  of  ‘images . 
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Visual  inspection  tasks 


Conventional  video  systems  for  inspection  (visual  in¬ 
spection)  of  products  and  specimens  nowadays  readily 
lend  themselves  to  connection  of  the  commercially  avail¬ 
able  sensors  with  an  evaluating  computer.  Such  systems  per¬ 
mit  the  following  tests  to  be  automatically  performed: 
completeness,  tests  on  simply  assembled  parts ,  .  damage  to 
components,  defects ■  in  workmanship ,  cleanness  tests,  pre¬ 
ferred  orientation  tests  on  grinding  operations,  cracking 
tests  on  cast  parts,  and  structure  tests  on  biological 
preparations  for  examination  under  the  microscope,  etc. 

It  is  however  a  prerequisite  for  the  automatic  carrying 
out  of  such  tasks  to  be  performed  with  the  television 
system  that  the  characteristics  to  be  evaluated  provide 
clear  contrast  conditions  through  optimising  the  illu 
ruination  and  object  preparation  and,  if  possible,  lend 
themselves  to  be  represented  by  a  binary  image. 


As  in  the  optical  representation  in  the  microscope,  two 
types  of  representation  are  also  distinguished  when  work¬ 
ing  with  television  sensor  systems:  working  with  through- 
light  illumination  and/or  vertical  illumination.  Under 
throughlight  illumination,  a  workpiece  is  represented 
in  its  contour  and  its  cut-outs-.  This  form  of  illumi¬ 
nation  mostly  permits  the  object  in  the  scene  to  be 
clearly  detected.  One  possible  application  of  the  through 
light  mode  is  the  inspection  of  plug  connector  strips 
or  the  position  of  the  contacts  in  it. 


Another  field  of  application  of  television  sensors  in  the 
throughlight  mode  is  the  inspection  of  glass  containers 
in  the  food  industry.  This  task  is  also  performed  in  the 
throughlight  mode,  but  it  can  only  be-  automated  in  pare, 
IdgccIuss  of  the  cons  id.  sir  a.  tie  clock  tintos* 


Figure  15  :  Plug  connector  strip  as  a  test  object 


Figure  16  :  Enlarged  representation  of  defective  terminals 
(away  from  center  shift) 
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Figure  17  :  Pressed-part  inspection 


The  maior  part  of  visual  inspection,  tasks  is  performed  in 

Tr,  t-uc,*  £ield,  very  different 
the  vertical  illumination  mode.  In  th.s  ,ie±a,  - 

visual  inspection  problems  have  been  analyzed  at  our  in¬ 
stitute.  The  most  frequently  occur ing  tasks  to  be  handled 

included: 

-  recognition  of  damage 

-  recognition  of  dirt  conditions 

-  detection  of  assembly  errors,  and 

-  detection  of  faulty  coating* 

inquiries  regarding  the  examination  of  products  for  damage 
came  from  a  wide  variety  of  production  sectors.  Such  in¬ 
quiries  came  e.g.  from  the  following  sectors: 

-  automotive  parts  manufacturing 

-  manufacturing  of  food  and  drug  containers 

-  pill  manufacturing 

-  screw  and  bolt  manufacturing 

-  manufacturing  of  electronic  components,  and 

-  manufacturing  of  wovens. , 

The  tasks  to  be  handled  -are  to  be  represented  with  the  aid 
of  the  following  pictures'  (Figures  18  -  21),  m  the  form 


of  extracts. 


Figure  19  :  Internal  thread,  with  'defects 


ro. 


Figure  20  :  Testing  a  chip  for  defects  (before  bonding) 
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r *  Figure  21  :  Cracking  and  shrinkage  test  on  piston  rings 
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Apart  from  tests  for  dcuaags ,  the.  cleanliness.  ri£  the  con¬ 
tainers  (bottles ,  cans ,  tubes,  etc-K  was  of  decisive  im¬ 
portance,  in  particular  in  the  field  of  the  food  and  drug 
industries.  The  use  of  television  sensor  systems ' in  these 
inspection  jobs  failed  in  this  field  of  application  because 
of  the  considerable  clock  times.. 

In  the  assembly  environment,  it  is  mainly  the  electronics 
industry  that  has  an  interest  in  an.  increased  use  of  tele¬ 
vision  sensor  systems.  The  main  inspection  j'H3S  in  this 
field  include  the  supervision  of  the  assembly  of  printed 
circuits  and  the  supervision  of  assembly  wo \  rz  of  plugs  and 
small  equipment. 

A  further  field  of  application  of  television  picture-pro¬ 
cessing  systems  is  in  the  field  of  surface  testing  of  coated 
plates.  On  the  strength  of  the  high  resolution  of  these 
sensor  systems  (2.56  gray  levels)  ,  they  are  in  this  field 
far  superior  to  the  human  worker,  who  can  cope  with  only 
about  20%  (50  to  60  gray  levels). 


Applications  in  metrology 

Press  engineering  has  made  considerable  progress  over  the 
.past  few  years.  The  use  of  multiple  press  tools  makes  it 
possible  today  to  completely  manufacture  increasingly  com¬ 
plex  parts  in  rapid  sequence.  In  contrast  to  this  highly 
advanced  production  method,  the  quality -^assuring  measures 
for  initial  sample  inspection  and  for  in-production  testing 
have  not  been  further  developed  to  the  same  extent.  In  ini¬ 
tial  sample  inspection  and  in  sampling  tests,  acceptance- 
test  times  from  one  to  three  days  f  carried  out  every  one 
or  two  hours,  are  the  rule’  for  a  full-time  inspector.  The 
high  costs  caused  by  the  inevitable  stop  times  are  a  burden 
for  the  "pfennig"  product  "pressed  part". 
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The  spectrum  of  parts  manufactured  in  the  press  shop  -is 
very  versatile.  It  includes  very  complicated  workpieces 
(Figure  22)  with  a  large  number  of  different  cut-outs, 
beads,  and  angular  sections,  but  also  workpxec.es  of  simple 
geometry,  which  can  be.  manufactured  in  very  large 
auantities * 


The  stroke  rate  of  the  .machines-  is  around  5G  strokes  per 
minute  in  complicated  parts,  and  around  180  strokes  in 
straightforward  parts.  Multiple  tools  are  the  rule. 

Although  the  tools  have  long  useful  lives,  there  are  impor¬ 
tant  dimensions  in  which  wear  leads  to  quality  problems  in 
further  working.  Another  problem  is  the  breaking  out.  of 
blanking  punches.  If  not  detected  in  time,  this  defect  will 
result  in  high  reject  costs. 


Figure  22  :  Pressed  part  with  complicated  cut  oui_s 
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If  an  analysis  is  made  of  the  measuring  and  supervision 
jobs,  performed  on  pressed  parts,  it  can  be  seen  that  a 


limited • number  of  partial  tasks  that  can  be  standardized 
occur  again  and  again.  These  are  known  to  thq  engineer , 
and  he  can  break  down  a  complex  inspection  job .into  these 
partial  tasks.  If  these  metrological  partial  tasks  are 
solved  in  terms  of  software  and  hardware,  it  will  be  pos¬ 
sible  to  automatically  inspect  pressed  parts  with  the  aid 
of  the  test  center  after  a  teach-in  phase  (Figure  23) . 

Problem-orientated  program  modules  are  established  for  such 
oartial  tasks  with  the  assistance  of.  the  built-in  ‘develop¬ 
ment  system.  Thesfe  will  then  be  assembled  in  a  task-related 
manner. 


Figure  23  :  Television  test  center  for  pressed  parts 
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Just  as  in  pressed  parts,  highly  accurate  .precis ioh-wgi> 
neering  workpieces,  also  pose  groat  problems  in  quality 
assurance.  To  test  the  bottom  plate  of',  a  watch  (Figure  24)  . 
today  takes  about  three  hours.  In  this  case ,  too,,  the  tele-v 
vision  measurement  method  can  bci  used  to  advantage.  Due 
to  the  extremely  high  demands  made  on  the  accuracy  of  such 
parts  it  is  however  necessary  to  increase  ‘-ho  rejCiu>ion 
of  optoelectronic  sensor  systems  by  the  inclusion  of  ..me¬ 
chanically  operating  scanning  tables-.  This  combination  per¬ 
mits  shorter  inspection  times,. and  a  quality-assurance 
system  can  thus  be  built  which  does  not  only  orientate  it 
seif  at  the  rather  doubtful  small  sampling  tests,  but  it 
receives  more  data  from  the  faster  operating  control  systems. 


Figure -24  :  Bottom  plate  of  a  watch 
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Problems  in  the  use  of  sensor  systems. 

•Extensive  basic  investigations,  have  been  conducted  in-  the 
field  of  digital  picture  processing  since  the  beginning 
of  the  seventies.  Supported  by  the  rapid  development  in 
the  semiconductor  and  computer  fields  and  also  by  the 
accompanying  cost  reduction  in  these  fields ,  it  has  been 
possible  to  build  picture-processing  systems  coupled  to 
computers  at  reasonable  costs.  But  the  task-specific 
software  development  is  still  very  cost-intensive  to  the 
present  day.  This  is  one  of  the  reasons  why  such' systems 
come  into  use  only  hesitantly. 

Further  reasons  which  hinder  the  ■■  application  of  such  sen¬ 
sor  systems  are: 

-  shading  effects. 

-  special  illumination  equipment 

-  too  low  clock  time  for  each  test 

-  sensitivity  of-  sensors,  and 

-  insufficient  contrast  conditions. 

The  shading  problems  at  least  call  for  a  hardware  cor¬ 
rection  of  the  picture  scene.  But  it  is  often  experienced 
that  such  a  correction  module  is  not  satisfactory.  Mea¬ 
surement  must  then  be  integral  and  the  correction  is  made 
through  software.  This  in  turn  partly  leads  .to  consider¬ 
ably  longer  test,  clock  times.  Another  not  insignificant 
aspect  is  the  demand  made  on  the  illumination  equipment. 
While  simple  plane  light  sources  are  sufficient  for  work¬ 
ing  in  the  throughlight  illumination  mode,  to  produce 
clear  contrast  conditions,  working  in  the  vertical  illu¬ 
mination  mode  partly  requires  the  use  of  rather  elciborcite 
special  light  sources.  These  special  lighting  facilities 
in  part  lead  to  a  considerable  increase  in  the  system 
price . 

The  test  stations  on  the  shop  floor  arc  partly  accomo¬ 
dated  in  a  rather  rough  environment,  TV  cameras  have  a 
very  sensitive  tube  and  mu-st  therefore  be  pro.tected  from-  - 
external  influences,  (vibration,  heat,  etc.) . 
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developed  for  specific  picture  enhancement  alpo-ith,. 
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Prospects 
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ores  1  -  use  of  diode-array  cameras 

growing  resolution  will  eutend  r ZlTo  dras- 

tions  of  picture-processing  systems.  As  a  re. 

ice  falls  the  applications  which  are  now  typical 

tic  price  falls,  one  .1  nodical  engineer- 

in  the  field  of  military  engineering  ~nd  medical  c  u 

th  extension.  Optp-electro- 

mg  will  ewpenen  productlon  environment 

nic  sensor  systems  wili  enter  tne  1  .  ^ 

1 nrae  scale.  The  tasks  to  be  expected  xn  qualiw 
°n  "  "  particularly  be  visual  inspection,  geometry 

"P::  e  su  ervislon,  inspection  of  assembly 
operations^  and  pattern  recognition  for  controlling  in- 
telligent  handling  devices.  $ 

the  teach-in  methods  which  are  still  nccos- 
Furthermore,  the  teacn  n  ter 

nary  at  present,  will  lergely  be  taken  over  by  conpu 
groups,  used  In  divisions  like  design,  development  ^ 

^  1  „  ran  CAM)  for  making  a  product, 

work  scheduling  (e.g.  CAD,  cami 
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It  1S  Obvious  that  the  development  in  production  engineer¬ 
ing  points  towards  even  more  rapid,  more  reasonably  priced 
more  reliable,  highly  automated  flexible  manufacturing  systems. 
A,  a  necessary  consequence,  the  demands  made  on  the  super 
vising  and  controlling  units  will  also  rise,  particularly 
with  respect  to  optical  units. 

Much  attention  Is  given  to  research  and  development  in  opti¬ 
ca!  sensor  systems,  both  in  the  united  states  and  m  .he  re 
derai  Republic  of  Germany.  However  technology  transfer  into 
industrial  applications  Is  definitely  lagging. 

This  then  is  the  starting  point  for  further  worm  in  tne  a 
?::l  of  .sensorica.,,  i.  the  integration  of  complex 

non-contact  measuring  systems  into  the  fields  of  qua-i 
gineer-ng.  With  the  increased  use  of  such  highly  automated 
sustains ,  additional  changes  Will  become  necessary  throughout 
the  field  of  quality  assurance,  especially  with  respec.  to 
organization.  These  problems  although  only  hxxus.  y 
in  this  paper,  form  a  fundamental  part  of  the  activities 

the  IPA. 

The  experience  gained  in  the  use  of  a  wide  variety  "!  »?'■  x  1 
sensors  (linear  diodes,  diode  arrays,  photosensitive  indivi¬ 
dual  elements,  complex  linear  systems,  etc)  has  rose 
in  a  state  of  development  at  the  IPA  that  makes  it  pose 
to  link  individual  sensor  types  to  the  modular  sof.wai.. 

The  outcome  is  a  flexible  unit,  which  can  be  easily  convcr 
The  outcome  ■«■=>  .  data  reduction , 

to  changing  parameters  (data  transfer  rax,  da 

under  the  field  «****-*£ 

so  far  been  somewhat  neglected.  This  snoura^be  a 
special  significance  in  the  aerospace  m  usun ,  • 

duction  of  high  quality  parts  is  essential. 


